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Abstract. While it is well known that a Turing machine equipped with the 
ability to flip a fair coin cannot compute more that a standard Turing machine, 
we show that this is not true for a biased coin. Indeed, any oracle set X may be 
coded as a probability px such that if a Turing machine is given a coin which 
lands heads with probability px it can compute any function recursive in X 
with arbitrarily high probability. We also show how the assumption of a non- 
recursive bias can be weakened by using a sequence of increasingly accurate 
recursive biases or by choosing the bias at random from a distribution with 
a non-recursive mean. We conclude by briefly mentioning some implications 
regarding the physical realisability of such methods. 
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1. Introduction and Motivation 

The Turing machine is well known to be a very robust model of computation. 
In almost all textbooks on the theory of computation, one can find a list of exten- 
sions to the Turing machine that offer it more primitive resources, such as extra 
tapes or nondeterminism, and yet do not give it the ability to compute any addi- 
tional functions. Amongst such resources it is not uncommon to find references to 
probabilistic methods such as coin tossing. 

These methods can be made precise with the introduction of the probabilistic 
Turing machine or PTM 3 . A PTM is a standard Turing machine with a special 
randomising state. When the machine is in this state, the transition to a new state 
is not governed by what is on the tape, but by a random event. A fair coin is tossed 
and the machine goes to the specified 1-state if the coin comes up heads and the 
0-state if it comes up tails. 

Unlike a Turing machine, a PTM will not necessarily return the same output 
when run multiple times on the same input. Care must therefore be taken in 
defining what it means for a function to be computed by a PTM. One way is to say 
that a PTM computes a given function, /, is that if given x as input, along with a 
measure of accuracy j £ N, it produces f(x) with probability at least 1 — By 
this definition, a function is computable by a PTM if and only if it can be computed 
with arbitrarily high confidence. Alternatively, we could relax this definition and 
say that a PTM computes / if and only if when given x as input, it produces f(x) 
with some probability greater than \. 

It is quite easy to see that with either definition, a PTM computes only the 
recursive functions. For any PTM P, there is a Turing machine T that simulates 
it. T simulates each branch of the computation in parallel and keeps track of their 
respective probabilities. T also keeps a table which associates outputs with their 
probabilities. When a branch halts returning some value y, T creates a new position 
in the table for y and stores the probability of that branch occurring. If a branch 
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has already halted with output y, T simply adds the new probability of producing 
y to the old value. After each update to the table, T checks whether the new value 
for y is greater than P's threshold (i or 1 — and halts returning y if this is 
so. In this way, T halts with output y if and only if P returns y with sufficient 
probability. 

This argument can also be extended to deal with more complicated probabilistic 
methods. For example, we could allow biased coins where the chance that heads 
comes up is some given rational number. We could even allow the bias to be any 
recursive real number (as defined in section[2]of this paper). In each case, T can still 
keep track of the probability of each computation branch and test to see whether 
an output occurs with enough probability to be deemed the output of the PTM. 

It is important to ask, however, what can be computed if non-recursive probabil- 
ities are used. In this paper, we show that allowing coins with non-recursive biases 
makes the above argument fail quite spectacularly. We first show that a PTM can 
compute arbitrarily accurate estimates to the bias on its coin and then strengthen 
this to computing arbitrarily many bits of the binary expansion of the bias. 1 From 
this, we reach several strong theorems about the power of PTMs, showing in partic- 
ular that there is a single PTM that acts as a universal o-machine: when equipped 
with a probability coding a given oracle, it simulates a given o-machine with that 
oracle on a given input to a given level of confidence. Thus, the addition of ran- 
domness to the resources of a Turing machine can certainly increase its computable 
functions. Only when the coins are restricted to recursive biases does it offer no 
additional power. 

In the remaining sections, we show two ways in which the same results are 
possible with slightly weakened resources. Specifically, we show how a sequence of 
rationally biased coins can be used, so long as the biases converge effectively to a 
non-recursive real or the biases are drawn at random from a distribution with a 
non-recursive mean. Finally, we point to some interesting physical applications in 
which these types of probabilistic methods seem to be consistent with Quantum 
mechanics. 



2. Approximating p to arbitrary accuracy 

The natural way to approximate the probability p that the coin will land heads, 
is to look at the average number of heads in n tosses. By the weak law of large 
numbers, this value (which we will denote p) approaches p as n approaches infinity. 
However, to approximate p effectively, wc need to know how fast this convergence is 
likely to be. This can be expressed by asking how many tosses are required before 
p is within a given distance of p with a given level of confidence. Specifically, we 
will ask for a method of calculating n such that when at least n tosses are made, 
\p ~ p\ < ^ with probability at least 1 — ~ for given j, k e N. 



^Since writing this paper, an article by Santos [5| has been brought to the authors' attention 
wherein a similar result is shown. However, it is our opinion that Santos' proof is incomplete, 
lacking an explanation of how the binary expansion of the probability can be computed from the 
rational approximations. In any event, we think the present account is useful for its further results 
and use of only elementary methods. 
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The probability distribution of possible values of p for a given value of n is a 
binomial distribution with mean p. The variance of p is given by 

(2.1) a 2 = P{1 - P) 

n 

This variance depends upon the unknown value of p, however since it has a 
maximum where p = \ , we can see that 

(2.2) a 2 < ]- 

With this upper bound for the variance, we can use the Chebyshev inequality 

2 

(2.3) Ve>0 P(\x - ii\ > e) < % 

to form an upper bound for the probability of error 

/ 1 \ 2 2fe 

(2.4) Vfc P^-p|>_j<_ 

Therefore, if we insist on a chance of error of at most ~, this can be achieved 
so long as 

n2k i 

(2.5) < — 

(2.6) n > V +2k - 2 

Thus, for each value of p E [0, 1] we can compute an approximation of p that 
is within an arbitrarily small distance of the true value with an arbitrarily high 
probability. More formally, 

Theorem 2.1. There is a specific PTM that, when equipped with a probability p, 
takes inputs j, k e N and outputs a rational approximation to p that is within of 
the true value with probability at least 1 — ~. 

Proof. The PTM simply tosses its coin 2 J+2fe ~ 2 times and returns the ratio of heads 
to tails. By the argument above, this approximation will suffice. □ 

This method of approximating a real number by successively accurate rational 
approximations can also be used to define a notion of which real numbers are 
computable by a (deterministic) Turing machine. For convenience, we say 

Definition 2.2. {x n } converges quickly to x if and only if \x n — x\ < ^ for all n. 

We can then define a notion of a recursive real. 

Definition 2.3. x G R is recursive if and only if there is a Turing machine that 
takes neNas input and returns x n £ Q, where {x n } converges quickly to x. 

The recursive reals given by this definition are well studied and include a great 
many of the reals actually encountered in mathematics, including all the algebraic 
numbers as well as tt and e. However, since there are uncountably many reals 
but only countably many Turing machines, it is clear that most of them are not 
recursive. If a PTM is equipped with one of these non-recursive reals as its proba- 
bility, then our algorithm of the previous section shows that in a certain sense, this 
PTM can compute this real — a feat that is impossible with a deterministic Turing 
machine. 
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However, there is still some room to question whether the PTM of Theorem 12. II 
actually computes its probability. Consider, for example, the following alternative 
definition of a recursive real. 

Definition 2.4. i e 1 is recursive if and only if there is a Turing machine that 
takes no input and outputs a sequence {x n } which converges quickly to x. 

This definition is evidently equivalent to the previous one when it comes to de- 
terministic Turing machines, but it is not immediately clear that the equivalence 
holds for PTMs. While the PTM of Theorem l2.1l can compute each approximation 
to x with arbitrary accuracy, it is not clear that a PTM could output an infinite 
sequence of approximations with them all being correct with arbitrarily high proba- 
bility. However, we now show that this can be achieved by requiring each successive 
event to be more and more probable. 

For a given minimum probability q for an entire infinite sequence of events oc- 
curring, we can set the probability of the i-th event occurring = q 2 . It follows 
that the chance of all events occurring is 

(2.7) \\q' ,,>■' q 

i=l 

In addition, we can consider the chance that all events in an infinite suffix of the 
sequence occur. The chance of all events after event iV occurring is 



(2.8) Y[ f ' = q £r = N+l2 ' = q 2 



i=N+l 

Thus, for each e > 0, there is a value of N for which the probability of all events 
after event N occurring is within e of 1. Therefore, the probability that some 
infinite suffix of these events will occur must be equal to 1. 

This construction can be applied in the case of our approximations to p. In 
particular, we can find a new value j' as a function of j and k which can then be 
substituted into our formula for the number of required coin tosses. 

1 / 1 ^ 2 " 



( 2 -9) 1 — —7 = 11 — 

y J 23 \ 23 

Using a Taylor expansion, we can see that for < x, y < 1 

x) v < 1 — xy 



(2.10) 


(1 


and thus 




(2.11) 


1 


1 T 

23' 


(2.12) 


1 

1 T 

23' 


(2.13) 


f 



< 1 2i+ k 

< j + k 
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Putting this all together, 



Theorem 2.5. There is a specific PTM that, when equipped with a probability 
p, takes input j £ N and outputs a sequence {pk} that converges quickly to p. 
Furthermore, with probability 1, there is some N such that \pk — p\ < for all 
k> N. 
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Proof. For each value in the sequence, the PTM simply tosses its coin 2- J+3fc ~ 2 times 
and returns the ratio of heads to tails. By the argument above, these approxima- 
tions will suffice. □ 

3. Computing the binary expansion of p 

The definitions of the previous section are not the only ways that the recursive 
reals can be defined. Instead of using converging sequences of rationals, we can 
use the original technique due to Turing [9] of using the base b expansion. For 
simplicity, we use the binary expansion and only consider those reals in the unit 
interval. 

Definition 3.1. x e R is recursive if and only if there is a Turing machine that 
takes n 6 N as input and returns b n , the n-th bit of the binary expansion of x. 

As before, we can rephrase this to speak of Turing machines that take no input: 

Definition 3.2. x e R is recursive if and only if there is a Turing machine that 
takes no input and returns the sequence {bn}, coresponding to the binary expansion 
of x. 

Both definitions run into an ambiguity in the case of dyadic rationals: those that 
can be expressed in the form For such numbers, there are two binary expansions 
so we adopt the convention of using the one containing an infinite number of O's. 

By extending our method for approximating p, we can also approximate the 
binary expansion of p. Unfortunately this will not be possible if p is a dyadic 
rational, so for now consider the case where it is not, and p thus has a unique 
infinite binary expansion in which both and 1 occur infinitely many times. To 
compute the binary expansion of p, we need a method that takes inputs j, I and 
gives us the value of bi with probability 1 — 53-. 

It may seem as though this can be achieved simply by computing pi+i and tak- 
ing its l-th bit, but problems arise when a run of consecutive O's or l's occurs 
around this point in the expansion. For instance, if we want the third bit and 
p4 = .01111111, then the true value of p could be as low as .01101111 or as high as 
.10001111 and we can thus be certain of none of the bits. By using the following 
algorithm, which we shall call A, we can overcome this problem. 

• k :=l 

• repeat 

— k := k + 1 

— compute pk (by tossing the coin 2-? +3fc ~ 2 times) 

— if pk < 1 and there are both a and a 1 between the l-th. and fc-th bits 
of the expansion of pk then output the l-th bit 

An analysis of A is made somewhat complex by the fact that it involves random 
events and does not always give the correct output, but for now we will just con- 
sider the most probable case where the probabilistically generated sequence {pk} 
converges quickly to p. We can see that there must be a value of k for which pk 
is less than one and has both a and a 1 between its l-th. and k-th bits, for if 
there were not then pk would either be approaching a dyadic rational or failing to 
converge — each of which would contradict our assumptions. Therefore, so long 
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as p is not a dyadic rational and {pk} converges quickly to p, B will always halt. 
When it does, the value of pk will be in the form 

(3.1) p k = in... M...106fc... 

or 

(3.2) /<;. ...ML.. 01 /',,... 

In either case, adding or subtracting a value smaller than ^ will not change 
any of the first I bits of pk and since p is within of pk, their first I bits must be 
identical. 

It is important to note, however, that while all runs of l's or O's within the 
expansion of p must come to an end, they can be arbitrarily long, so the running 
time of A depends upon the value of p. If the i-th bit of the expansion of p is 
followed by a run of m identical bits, then we must compute I + m values of pk, 
requiring at most 2-?+ 3i+3m ~ 1 coin tosses. 

What about those cases where {pk} does not converge quickly to pi This can be 
for two different reasons — cither it converges to p, but not as quickly as required 
or it does not converge to p at all. The first of these cases occurs with probability 
~ and while it cannot cause B to fail to halt, it may well cause an incorrect output. 
The second case occurs only with probability 0, and may either cause an incorrect 
output or non-termination. 

Theorem 3.3. There is a PTM that implements A. Equipped with any non-dyadic 
probability p, it takes positive integers j and I, outputting the l-th bit of the binary 
expansion of p with probability greater than 1 — ~. The probability that it returns 
an incorrect answer is less than ^j, while the probability that it does not terminate 
is 0. 

Proof. Immediate □ 

We can also modify A to form Aao which takes only j as input and outputs the 
entire expansion of p. In this case it outputs the l-th digit when it has output all 
prior digits and has found a value of pk with a and a 1 between its l-th and fc-th 
digits. Aqo uses the high likclyhood of {pk} converging quickly to p to greater effect 
than A, by generating the entire expansion with arbitrarily high probability 

Theorem 3.4. There is a PTM that implements A^. Equipped with any non- 
dyadic probability p, it takes a positive integer j , outputting the entire binary ex- 
pansion of p with probability greater than 1 — ^j. The probability that it outputs 
finitely many incorrect bits is less than ~, while the probability that it outputs 
infinitely many incorrect bits or outputs only a finite number of bits is 0. 

Proof. Immediate □ 

4. Using the binary expansion of p as an oracle 

In 1939, Alan Turing ^D] introduced a very influential extension to his theoretical 
computing machines. Turing's o-machines are standard Turing machines combined 
with a special 'oracle', which can answer questions about a particular set of natural 
numbers, called its oracle set. Like a PTM, an o-machine has a special query 
state and two answer states, but instead of the answer being given randomly, it 
corresponds to whether a certain number is in the oracle set. To specify the number 
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whose membership is being questioned, a special symbol /x is inscribed twice on the 
tape and the number of squares between each inscription of n is taken as the 
query to the oracle. Depending on which oracle set is given, an o-machine can 
compute different classes of functions, and they thus give rise to a notion of relative 
computability. 

Corresponding to an o-machine with oracle X we can construct a PTM with 
probability px where the n-th digit of the binary expansion of px is 1 if n £ X 
and otherwise. A PTM equipped with px can perform all basic operations of a 
Turing machine, as well as determining whether n£l for any n. It can do this by 
simulating B m in parallel with its main computation, storing the bits of p produced 
by Boo and examining them when needed. If it needs to test whether n <E X and 
has not yet determined b n , it simply waits until this is found. 

In the cases where px is a dyadic rational this method will not work, but since 
X will be recursive, there is a probabilistic Turing machine that can simulate such 
an o-machine without using any probabilistic methods at all. In this way, these 
methods suffice to simulate any o-machine. 

Theorem 4.1. For any o-machine M with oracle X, there is a PTM Pm equipped 
with probability px that when given the same inputs plus one additional input j, 
Pm produces the same output as M with probability greater than 1 — i. 

Proof. Immediate □ 

Since all functions of the form / : N n — ► N m or / : N" — » M m are computable by 
some o-machine, we can see that there are probabilities that would allow PTMs to 
compute any such functions. 

Corollary 4.2. For any function f : N™ — ► N m or f : Nn — > M. m , there exists 
a PTM that when given inputs j, x\ , . . . , x n produces f(xi, . . . , x n ) with probability 
greater than 1 — ~, produces incorrect output with probability less than and 
diverges with probability 0. 

Since these natural and real numbers can be used to code other mathematical 
objects, this set of PTM computable functions includes a vast number of interesting 
mathematical functions. Given an appropriately biased coin, a PTM could decide 
the halting problem or the truths of first order arithmetic. 

Finally, just as there is a single universal Turing machine which can take the code 
of a Turing machine as input and simulate it, so there is a universal o-machine which 
takes the code of an arbitrary o-machine and simulates it so long as it is equipped 
with the oracle of the machine being simulated. A similar job can be performed 
by a specific PTM, provided that the o-machine to be simulated does not have an 
oracle set that would be encoded as a dyadic rational. As such o-machines can only 
compute recursive functions, this is not a great concern. 

Theorem 4.3. There is a specific PTM Pjj that takes inputs j, n,m E N and when 
equipped with any non-dyadic probability px , Pu computes the result of applying the 
o-machine with oracle X and index n to the input m, producing the correct output 
with probability at least 1 — -y. 



Proof. Immediate 



□ 
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5. Getting by with increasingly accurate biases 

These same results can all be realised without the need for a coin with an infin- 
itely precise bias. Instead, consider a variant of the PTM which is given a succession 
of coins {c„} where the n-th coin is used for the n-th toss. If the probability of 
c„ coming up heads is given by the rational probability p n and {p n } converges 
quickly to some arbitrary real p, then all of the above results hold with only minor 
modifications. 

If we once again approximate p using the average number of times heads comes 
up in n tosses, we find that the mean of p is no longer p, but /j,, where 

(M, 2^ + 1 

n n n n 

By a similar argument, we find the lower bound for /z, and see that 

(5.2) p- - < n<p+- 

n n 

The variance is now given by 

(5.3) CT 3 = £_i__^__)<J_ 

^ n 2 4n 

i—l 

We can now once again use the Chebyshev inequality to form an upper bound 
for the probability of error. If we set n = 2 J+2fe (which is 4 times higher than the 
value of n used previously), we see 

P(\P-»\<^l) > 1-^ 

P(\P-P\<^) * 

And so this new value of n suffices in this case. Replacing all later references to 
2 j+2fe-2 with 2 j+2fe and rc f crcnccs t0 2J'+3*=-2 w ith 2 j+3k , all the theorems follow. 

It is also easy to see that we could relax our constraint of sequence that converges 
quickly to a sequence that converges at a rate bounded by some recursive function. 
Then we could calculate a subsequence of these coins whose probabilities would 
converge quickly and use that. 

6. Getting by with randomly chosen biases 

Another way that we can avoid the need for a coin with an infinitely accurate 
bias is via a probability distribution of finitely accurate biases. As in the previous 
section, we use a sequence of coins {c„} where the n-th coin is used for the n-th 
toss. This time however, the bias on each coin will be chosen with an independent 
random trial from a fixed probability distribution. We will see that so long as 
the mean of this distribution is a non-recursive real, access to this randomisation 
extends the PTM's powers. Specifically, it can compute the binary expansion of 
the mean with arbitrarily high confidence. 

We first consider the case of a discrete probability distribution, where the prob- 
ability of choosing the bias Xi G [0, 1] is denoted by P(xi is chosen). To generate 
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the value of the n-th coin toss, we must combine the process of randomly choosing 
a bias with the process of tossing a coin with that bias. Let z be a random variable 
representing the result of the coin toss, equalling 1 if the coin lands heads and if 
tails. From the rules of conditional probability, 

(6.1) P(z — 1) = P(z = l\xi is chosen)P(xi is chosen) 

i 

= ^^XiP{xi is chosen) 

i 

= Px 

The same is true if we use a continuous distribution p(x). In this case 

(6.2) P(z = 1) = / P(z = l\x is chosen)p(a; is chosen)dx 

Jo 

l 

xp(x is chosen)da; 

= P-x 

In either case P(z = 0) = 1 — fi x . Thus, the combined process of randomly 
choosing a bias between and 1 from any distribution with mean /i x and then 
flipping the appropriate coin is equivalent to the process of flipping a single coin 
with bias p x . From this it is clear that one can determine the binary expansion 
of p x with arbitrary confidence using the methods of sections [^HH Indeed, all the 
results of those sections hold for this modified type of PTM without the need for 
any additional coin tosses. 

In the case of discrete distributions, it is interesting to consider how p x could 
be non-recursive. Recall that for a discrete distribution, the mean is defined by 
J2i XiP(xi) and that the recursive reals are closed under finite sums and products. 
Thus, if the distribution is finite, the only possibilities are that at least one of the 
possible biases is non-recursive or at least one of the associated probabilities is 
non-recursive. For infinite discrete distributions there is the additional possibility 
of one or both of the sequences {xi\ and {P(xi)} being non-recursive despite all 
the individual elements being recursive. 

Therefore, this method of randomly choosing a bias from a given distribution 
and then flipping a coin with that bias allows a PTM to exceed the power of a 
Turing machine without relying upon coins with infinitely precise biases. 

7. Conclusions 

Over the course of this paper, we have shown three ways to implement an abstract 
oracle by tossing biased coins. This was achieved by demonstrating a computational 
equivalence between o-machines and three different classes of PTM. These results 
show that it is very careless to say that randomness does not increase the power of 
the Turing machine. While this is true of fair coins and recursively biased coins, 
they form only a set of measure zero in the space of all possible biased coins. Indeed, 
if a bias is chosen completely at random (from a uniform distribution over [0, 1]) 
then with probability one, it would be non-recursive and thus extend the powers of 
a Turing machine that had access to it. 

This is not only of mathematical interest, but is particularly significant in the 
study of what is physically computable. There has been continued interest over 
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the years about whether some form of o-machine might be physically realisable 
E] • A simple way to go about implementing an oracle would be to measure 
some quantity, such as the distance between two particles, with finer and finer 
accuracy. If this distance happened to be a non-recursive real, we could then use 
the methods of section [3] to compute the binary expansion and use this as an oracle 
set. However, such methods based on measuring continuous quantities could quickly 
run into fundamental limits of Quantum Mechanics, especially if there exists some 
fundamental lengthscale such as that of the Planck scale as demanded by some 
theory of Quantum Gravity. 

Using randomness provides an alternative that does not run afoul of these limita- 
tions. It allows one to measure an underlying continuous quantity with a sequence 
of discrete measurements that do not individually become increasingly accurate. It 
is the increasing total number of measurements that provides the accuracy, so no 
particular measurement needs to be more accurate than the quantum limits. 

In fact, quantum mechanics even suggests a means of simulating such biased 
coin tosses. A qubit is a generic name for a quantum system that has two possible 
states and when measured, is seen to take on one of these states randomly [Hj . Each 
state of a qubit has an associated probability amplitude, which is a complex number 
that defines the probability that the system will be found in that state. These 
probability amplitudes are allowed to be arbitrary complex numbers having moduli 
less than one and thus the induced probabilities, which are squares of the moduli, 
are arbitrary reals. A qubit therefore seems to be a physical implementation of an 
arbitrarily biased coin. 

There is, however, an important dissimilarity in that while a biased coin can 
be flipped as many times as one wishes, a qubit is destroyed once its state is 
determined. Furthermore, by the no cloning theorem of quantum mechanics 11 , 
we cannot get around this destructive measurement by making perfect copies of the 
qubit. 

However, the technique of section seems to offer a way out. If there is any 
method of creating qubits with biases chosen randomly around some non-recursive 
mean, then this method implements a non-recursive oracle. Since the non-recursive 
values this mean could take form a set of measure one in the space of all possible 
biases, this appears quite plausible and it would seem to require an independent 
physical principle to force all such methods to pick out only recursive means. 

If we furthermore wish to harness this non-recursive power to compute some 
particular non-recursive function, we need to know more about the non-recursive 
mean around which our biases are generated. For instance, we could try to create a 
PTM for deciding whether a given formula of the predicate calculus is a tautology 
by using a mean that codes the set of halting Turing machines, or even by using 
the halting probability f2, described by Chaitin 1., in the setting up of a qubit 

However, it appears to be very difficult to generate biased qubits around such a 
known mean. Consider some controllable variable A (such as the amount of time 
an electron is exposed to a magnetic field) involved in creating the probability 
amplitude a(A) for a qubit state and let us suppose that we could generate this 
controllable variable in some distribution P\ with the appropriate mean. Even 
then, we would still have further difficulties to overcome as the relationship between 
the bias of a qubit, represented by |a(A)| 2 , and the controllable variables that 
determine it is inevitably non-linear. It is then not sufficient to control the mean 
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of the controlled variables A: we must also precisely know the details both of 
their distributions P\ and of the functions a(A), which can and will be affected by 
generally uncontrollable quantum decoherence, to obtain the mean of the quantum 
probabilities through their distributions P a , 



and it seems quite unlikely that all of these would be possible. 

The use of biased coins to compute more than the Turing machine is certainly 
of some physical interest and, although it is not yet clear how it could be physi- 
cally harnessed to increase our computational abilities, the close connections with 
quantum theory suggest a potential for further study. 
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